Click any tag below to further narrow down your results
Links
OpenAI released the Decisions API in public beta, which makes classification and decision-making tasks roughly 10 times faster than their Responses API. The tradeoff is higher cost per request and currently no token caching support, which some developers find economically unattractive for certain use cases.
- Decisions API processes text and image inputs to return typed answers (probabilities, choices from sets, rubric scores) about 10x faster than the Responses API, with median response times around 0.3-0.8 seconds on typical connections
- Accuracy is comparable to Luna (the model used in Responses API) but slightly faster and more variable in performance, though the practical difference is negligible
- No token caching is currently available, making the API more expensive per request than alternatives for repeated classification tasks—a limitation developers hope gets addressed soon
Jev is a new AI model that only outputs structured data instead of human language, making it dramatically faster (70-500ms vs seconds) and enabling real-time applications like playing Doom. The author argues this speed advantage could become a new computational primitive for AI, though he suspects competitors can replicate it using simpler inference tricks on existing models.
- Jev generates all structured output in a single forward pass instead of token-by-token, achieving 70-500ms response times compared to seconds for standard LLMs, fast enough to play real-time video games.
- The speed advantage likely doesn't require novel model architecture—you can achieve similar results by prefilling responses and generating only one constrained token with existing LLMs, suggesting Jev lacks a substantial technical moat.
- Structured output could unlock entirely new use cases beyond chatbots by injecting "100ms worth of dirt-cheap intelligence" at decision points throughout applications, though Jev won't match frontier LLMs in raw capability.